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Art Unit: 2629 

DETAILED ACTION 

It would be of great assistance to the Office if all incoming papers pertaining to a 
filed application carried the following items: 

1 . Application number (checked for accuracy, including series code and 
serial no.)- 

2. Group art unit number (copied from most recent Office communication). 

3. Filing date. 

4. Name of the examiner who prepared the most recent Office action. 

5. Title of invention. 

6. Confirmation number (See MPEP § 503). 

Status of Claims 

1 . Claims 1-14 are pending. 

Response to Arguments 

2. Applicant's arguments filed on 09/17/2009 have been fully considered but they 
are not persuasive. 

3. As to Applicant's argument that Ohba fails to teach or suggest "means for 
executing predetermined processing when the detecting means detects the virtual 
touches that are made to a predetermined number of touch points in a predetermined 
order," the Examiner respectfully disagrees. In fact, Ohba does disclose means for 
executing predetermined processing (CPU 10 of fig. 2 executes a determined process 
when a certain process determining image is selected, [0071]) when the detecting 
means detects the virtual touches (the amount of movement of operators hand in a 
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particular area, [0069]) tliat are made to a predetermined number of touch points in a 
predetermined order (as shown in fig. 7, the menu image has a hierarchic structures, so 
in order to select "process22", the user has to perform virtual touches in the following 
order: first "menu" then "select2" and then "process22", [0070]). 

4. As to Applicant's argument that Ohba fails to teach or suggest that "the analyzing 
means analyzes the video image when each of the plurality of images, that prompts the 
input, is displayed, and detects that an area of the video image, corresponding to the 
selected area displayed in the manner different from others, includes a predetermined 
image; and a predetermined function is executed when production of the image that 
prompts the input, and the detection that the corresponding area includes the 
predetermined image, are performed a predetermined number of times," the Examiner 
respectfully disagrees. In fact, Ohba does disclose that the analyzing means (difference 
value detector 107 of fig. 4) analyzes the video image when each of the plurality of 
images (each menu of hierarchic structure, [0070]), that prompts the input, is displayed 
(the difference value calculated is indicative of one movement of operator's hand in the 
area in which the menu image is displayed, [0073]), and detects that an area of the 
video image, corresponding to the selected area displayed in the manner different from 
others (the difference is represented by change in color or the gradation of 
transparency, [0075]), includes a predetermined image (when cumulative sum exceeds 
threshold, controller determines that menu item has been selected, [0076]); and a 
predetermined function is executed (CPU executes determined process, [0071]) when 
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production of tine image tliat prompts tine input, and tine detection that tlie corresponding 
area includes tlie predetermined image, are performed a predetermined number of 
times (as shown in fig. 7, in order to execute "process22" the user has to got through 
hierarchical menu in following order: first "menu" then "select2" and then "process22", 
therefore the detection of touch has to be performed at least three times). 

5. As to Applicant's argument that Ohba does not teach or suggest determining 
whether or not movements according to a predetermined order in a plurality of points 
are detected, and displaying one point among a plurality of points in a manner different 
from others (in order to indicate a point to be touched next), the Examiner respectfully 
disagrees. In fact, Ohba does disclose determining whether or not movements 
according to a predetermined order in a plurality of points are detected (as shown in fig. 
7, in order to execute "process22" the user has to got through hierarchical menu in 
following order: first "menu" then "select2" and then "process22", therefore the detection 
of touch has to be performed at least three times), and displaying one point among a 
plurality of points in a manner different from others (the difference is represented by 
change in color or the gradation of transparency, [0075]), and it is Nakamura who 
discloses displaying one point among a plurality of points in a manner different from 
others in order to indicate a point to be touched next (such as monkey reply game 
disclosed in Nakamura, col. 30, lines 50 - 56). 
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6. As to Applicant's argument tliat Nal^amura does not teach or suggest displaying 
touch points connected sequentially subjected to the virtual touches, the Examiner 
respectfully disagrees. Ohba is the one who discloses detecting virtual touches using a 
camera, the reference of Nakamura is used to only show sequential connection of touch 
points, and it is the combination of these two references that would have been obvious 
to one of ordinary skill in the art at the time of the invention to teach applicant's claimed 
invention. 

Claim Rejections - 35 USC § 102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

8. Claims 1, 3, 6 - 8 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Ohba (US 2002/0097247). 

As to claim 1, Ohba discloses an information processing system, comprising: 
means for producing a computer image (image processor 2 of fig. 2, [0068]) that 
prompts a player to virtually touch a plurality of touch points (a menu comprising layers 
of pull down menus shown in fig. 7, [0070]); means for accepting input of a video image 
of the player (image input device 101 of fig. 3) picked up by image pickup means (video 
camera 1 of fig. 1); display control means for superimposing the video image and the 
computer image on one another and causing a display device to display a 
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superimposed image (superimposing image generator 106 of fig. 3); means for 
analyzing the video image during display of tine computer image (difference value 
detector 107 of fig. 3) and detecting a virtual touch to any of the plurality of touch points 
(object controller 105 determines touch [0075]); and means for executing predetermined 
processing (CPU 10 of fig. 2, [0070]) when the detecting means detects the virtual 
touches that are made to a predetermined number of touch points in a predetermined 
order (when appropriate process is selected form a pull-down menu, [0070]). 

As to claim 3 (dependent on 1 ), Ohba discloses the information processing 
system wherein: the image producing means sequentially produces computer images 
(menu items shown in fig. 7, [0070]) including navigation information indicating one 
touch point to be touched next (such as one SELECT point or PROCESS point of fig. 7); 
and the means for executing predetermined processing executes the predetermined 
processing (CPU 10 executes determined process, [0071]) when the detecting means 
detects that the virtual touches have been made for the respective computer images 
including the navigation information that is sequentially produced [0070 - 0071]. 

As to claim 6, Ohba discloses an entertainment system, comprising: means for 
producing a computer image including an image for specifying a plurality of areas 
(image processor 2 of fig. 2); means for accepting input of a video image (image input 
device 101 of fig. 3) picked up by image pickup means (camera 1 of fig. 1); display 
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control means for superimposing tine video image and tine computer image on one 
another and causing a display device to display a superimposed image (superimposing 
image generator 106 of fig. 3); and means for analyzing the video image by referring to 
the computer image (difference value detector 107 of fig. 3), wherein: the image 
producing means selects one area of the plurality of areas in a predetermined order 
(menu area of fig. 7), and sequentially produces a plurality of images (SELECT1 , 
SELECT2 and SELECT3 of fig. 7) each of which prompts the input [0070] and is 
obtained by displaying the selected area in a manner different from others (one of the 
SELECT images is highlighted when selected, [0070]); the analyzing means analyzes 
the video image when each of the plurality of images, that prompts the input, is 
displayed, and detects that an area of the video image, corresponding to the selected 
area displayed in the manner different from others, includes a predetermined image (the 
object controller determines the color of menu image according to the difference value, 
[0070 - 0071 and 0075 - 0076]); and a predetermined function is executed when 
production of the image that prompts the input, and the detection that the corresponding 
area includes the predetermined image, are performed a predetermined number of 
times (as shown in fig. 7, the user has to go through nested menu by pressing at lest 
three touch point in order for a process to execute, [0070 - 0071]). 

As to claim 7, Ohba discloses an information processing system input accepting 
method, comprising: producing a computer image that prompts a player to virtually 
touch a plurality of touch points (producing nested menu shown in fig. 7); accepting 
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input of a video image of the player (using image input device 101 of fig. 3) picked up by 
image pickup means (camera 1 of fig. 1); analyzing the video image (using difference 
value detector 107 of fig. 3) while superimposing the video image and the computer 
image on one another (using superimposing image generator 106 of fig. 3) and causing 
a display device to display a superimposed image (using display controller 108 of fig. 3), 
and sequentially detecting virtual touches to any of the plurality of touch points 
(sequentially touching nested menu shown in fig. 7, [0070]); and accepting the virtual 
touches as predetermined input when a predetermined number of touch points are 
touched in a predetermined order by the virtual touches (as shown in fig. 7, the user has 
to go through nested menu by pressing at lest three touch point in order for a process to 
execute, [0070-0071]). 

As to claim 8, Ohba discloses a recording medium (28 of fig. 2) on which a 
computer program for accepting input is recorded, the computer program causing a 
computer to execute processing of: producing a computer image that prompts a player 
to virtually touch a plurality of touch points Ohba; accepting an input of a video image of 
the player (using image input device 101 of fig. 3) picked up by image pickup means 
(camera 1 of fig. 1); analyzing the video image (using difference value detector 107 of 
fig. 3) while superimposing the video image and the computer image on one another 
(using superimposing image generator 106 of fig. 3) and causing a display device to 
display a superimposed image (using display controller 108 of fig. 3), and sequentially 
detecting virtual touches to any of the plurality of touch points (sequentially touching 
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nested menu shown in fig. 7, [0070]); and accepting tine virtual touches as 
predetermined input when a predetermined number of touch points are touched in a 
predetermined order by the virtual touches (as shown in fig. 7, the user has to go 
through nested menu by pressing at lest three touch point in order for a process to 
execute, [0070-0071]). 



Claim Rejections - 35 USC § 103 

9. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill In the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner In which the Invention was made. 

1 0. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 
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1 1 . Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ohba in 
view of Lee et al. (US 6,1 60,899). 

As to claim 2 (dependent on 1), Oliba discloses the information processing 
system, wherein the detecting means detects the virtual touch when an object within the 
video image overlaps any one of the plurality of touch points within the computer image 
[0070-0071]. 

Ohba fails to disclose that virtual touch is detected when an object having a 
specific color and worn by the player. 

In the same field of endeavor, Lee discloses detecting virtual touch when an 
object is worn by the player (ring of fig. 4A, col. 4, and lines 3 - 8). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to incorporate an object worn by the player in to the system of 
Ohba, so as to avoid erroneous recognition (Lee, col. 3, lines 63 - 67) 

Ohba in vie of Lee do not disclose the object worn by the player having a specific 
color. However, it would have been obvious to one of ordinary skill in the art at the time 
of the invention to associate a specific color with the ring object of Ohba in view of Lee, 
since Lee discloses that the system can also detect the color, such as flesh color (Lee, 
col. 3, lines 23-33). 
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12. Claim 9-14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ohba in view of Nal<amura et al. (US 7,331 ,856). 

As to claim 4 (dependent on 1), Oliba discloses the information processing 
system, wherein: the image producing means (image processor 2 of fig. 2) sequentially 
produces computer images (nested menu of fig. 7); and the means for executing 
predetermined processing (CPU 10 of fig. 2) executes the predetermined processing 
when the detecting means detects that the virtual touches have been made [0070 - 
0071] for the respective computer images including the navigation information that is 
sequentially produced (nested menu of fig. 7). 

Ohba fails to disclose an image each indicating two touch points to be touched 
next and detecting virtual touches have been made simultaneously on the two touch 
points. 

In the same field of endeavor, Nakamura discloses an image each indicating two 
touch points to be touched next (when a plurality of players present) and detecting 
virtual touches have been made simultaneously on the two touch points (col. 25, line 55 
-col. 26, line 3). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the device of Ohba to include features disclosed by 
Nakamura, so as to provide an interactive experience to a plurality of uses of the 
system. 
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As to claim 5 (dependent on 1), Ohba discloses tine information processing 
system, wherein: tlie image producing means (image processor 2 of fig. 2) produces the 
computer image including navigation information (nested menu of fig. 7); and the means 
for executing predetermined processing (CPU 10 of fig. 2) executes the predetermined 
processing when the detecting means detects that the virtual touches have been made 
according to the navigation [0070 - 0071]. 

Ohba fails to disclose navigation information indicating an order of touches to be 
made to the touch points 

In the same field of endeavor, Nakamura discloses navigation information 
indicating an order of touches to be made to the touch points (col. 30, lines 50 - 56). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the device of Ohba to include features disclosed by 
Nakamura, so as to provide an interactive experience to a user of the system. 

As to claim 9, Ohba discloses an information processing system, comprising; 
means for producing a computer image that prompts a player to virtually touch a 
plurality of touch points (image processor 2 of fig. 2); means for accepting an input of a 
video image of the player (image input device 101 of fig. 3) picked up by image pickup 
means (camera 1 of fig. 1); display control means for superimposing the video image 
and the computer image on one another and causing a display device to display a 
superimposed image (superimposing image generator 106 of fig. 3); means for 
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analyzing the video image during display of the computer image and detecting virtual 
touches to any of the plurality of touch points (difference value detector 107 of fig. 3); 
and the detecting means detects the virtual touches that are made sequentially to a 
predetermined number of touch points in a predetermined order (as shown in fig. 7, the 
user has to go through nested menu by pressing at lest three touch point in order for a 
process to execute, [0070 - 0071]). 

Ohba falls to disclose object displaying means for displaying an object that 
connects the touch points sequentially subjected to the virtual touches. 

In the same field of endeavor, Nakamura discloses an object displaying means 
(operation device 80, col. 35, lines 32 - 43) for displaying an object that connects the 
touch points sequentially subjected to the virtual touches (a large arrow which connects 
points to be touched, col. 35, lines 32 - 43). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the Invention to modify the device of Ohba to include features disclosed by 
Nakamura, so as to provide an interactive experience to a user of the system. 

As to claim 10 (dependent on 9), Ohba discloses the information processing 
system, wherein a virtual touch is made to any one of the touch points (menu points of 
fig. 7), but fails to disclose that the object displaying means displays the object that 
connects any one of the touch points to another touch point previously subjected to the 
virtual touch. 
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In the same field of endeavor, Nakamura discloses that the object displaying 
means (operation device 80 ) displays the object that connects any one of the touch 
points to another touch point subjected to the virtual touch (a large arrow which 
connects points, col. 35, lines 32 - 43). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the device of Ohba to include features disclosed by 
Nakamura, so as to provide an interactive experience to a user of the system. 

Ohba in view of Nakamura fails to disclose another point is previously subjected 
to a virtual touch; however, it would have been an obvious design choice alternative to 
connect the touch points In any order and would require an artisan only a routine skill in 
the art. 

As to claim 11 (dependent on 9), Ohba discloses the information processing 
system, wherein the virtual touches are made to a predetermined number of touch 
points in a predetermined order [0070 - 0071], but fails to disclose the object displaying 
means displays the object that connects the touch points sequentially subjected to the 
virtual touches. 

In the same field of endeavor, Nakamura discloses an object displaying means 
(operation device 80, col. 35, lines 32 - 43) for displaying an object that connects the 
touch points sequentially subjected to the virtual touches (a large arrow which connects 
points to be touched, col. 35, lines 32 - 43). 
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Therefore, it would liave been obvious to one of ordinary skill in the art at the 
time of the invention to modify the device of Ohba to include features disclosed by 
Nakamura, so as to provide an interactive experience to a user of the system. 

As to claim 12 (dependent on 9), Ohba discloses the information processing 
system, but falls to disclose that the object that connects the touch points comprises a 
line imitating light. 

In the same field of endeavor, Nakamura discloses that the object that connects 
the touch points comprises a line imitating light (an arrow on display inherently emits 
light, col. 35, lines 30-43). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the device of Ohba to include features disclosed by 
Nakamura, so as to provide an interactive experience to a user of the system. 

As to claim 13, Ohba discloses a recording medium (28 of fig. 2) on which a 
computer program for accepting an input is recorded, the computer program causing a 
computer to function as: means for producing a computer image that shows a player a 
plurality of touch points to be virtually touched (image processor 2 of fig. 2); means for 
accepting Input of a video Image of the player (image input device 1 01 of fig. 3) picked 
up by image pickup means (camera 1 of fig. 1); display control means for 
superimposing the video Image and the computer image on one another and causing a 
display device to display a superimposed image (superimposing image generator 106 of 
fig. 3); means for analyzing the video image during display of the computer Image and 
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detecting virtual touches to any of the plurality of touch points (difference value detector 
107 of fig. 3); and the detecting means detects the virtual touches that are made 
sequentially to a predetermined number of touch points in a predetermined order (as 
shown in fig. 7, the user has to go through nested menu by pressing at lest three touch 
point in order for a process to execute, [0070 - 0071]). 

Ohba fails to disclose object displaying means for displaying an object that 
connects the touch points sequentially subjected to the virtual touches. 

In the same field of endeavor, Nakamura discloses an object displaying means 
(operation device 80, col. 35, lines 32 - 43) for displaying an object that connects the 
touch points sequentially subjected to the virtual touches (a large arrow which connects 
points to be touched, col. 35, lines 32 - 43). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the computer program of Ohba to cause the computer to 
function according to features disclosed by Nakamura, so as to provide an interactive 
experience to a user of the system. 

As to claim 14, Ohba discloses an input interface controlling method for an 
information processor having a computer image producing part, an image input part, a 
superimposed image producing part, and a determining part, the method comprising the 
steps of: producing, by the computer image producing part, a computer image that 
shows a player a plurality of touch points to be virtually touched (nested menu of fig. 7); 
accepting, by the image input part (image input device 101 of fig. 3), an input of a video 
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image of tine player picl^ed up by image picl<up means (camera 1 of fig. 1); 
superimposing, by the superimposed image producing part (superimposing image 
generator 107 of fig. 3), the video image and the computer image on one another and 
causing a display device to display a superimposed image (using display controller 108 
of fig. 3): analyzing, by the determining part, the video image during display of the 
computer image (using difference value detector 107 of fig. 3) and detecting virtual 
touches to any of the plurality of touch points [0070 - 0071]. 

Ohba fails to disclose displaying, by the computer image producing part, when 
the virtual touches are detected to have been made sequentially on a plurality of touch 
points, an object that connects the touch points sequentially subjected to the virtual 
touches. 

In the same field of endeavor, Nakamura discloses displaying, by the computer 
image producing part (operation device 80, col. 36, line 40), when the virtual touches 
are detected to have been made sequentially on a plurality of touch points, an object 
that connects the touch points sequentially subjected to the virtual touches (trace of 
indicated positions, col. 36, lines 30 - 40). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the device of Ohba to include features disclosed by 
Nakamura, so as to provide an interactive experience to a user of the system. 
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Conclusion 

13. THIS ACTION IS MADE FINAL. Applicant is reminded of tine extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dmitriy Bolotin whose telephone number is (571 )270- 
5873. The examiner can normally be reached on Monday-Friday, 8:00 AM - 5:00 PM 
EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amare Mengistu can be reached on (571)272-7674. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding tine status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

ID. B.l 

Examiner, Art Unit 2629 

/Amare Mengistu/ 
Supervisory Patent Examiner, Art Unit 2629 



